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Abstract 

Books and courses on research methods, statistics, or both, often necessarily focus on 
one topic at a time. This compartmentalized approach prevents students from seeing 
the big picture. To address this shortcoming, I developed an exercise through which 
students experience the whole research process in a single class period. From posing a 
question, through devising a method, collecting and analyzing data, to writing the re¬ 
port, students conduct all aspects of a basic research project. After the experience, stu¬ 
dents reported a significant increase in their understanding of the research process. In 
addition, students recommended using the exercise in future classes and at other univer¬ 
sities. I end by discussing various ways that the exercise can be adapted to other class 
situations and content areas. 
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Tell me and I'll forget. Show me and I might remember. Involve me and I'll under¬ 
stand. - Unknown 

The chapters in most texts, whether books on research methods, statistics, or both, focus 
on one topic at a time. A typical introductory research methods book has chapters on de¬ 
scriptive methods, experimental methods, ethics and single case and quasi-experimental 
designs (Shaughnessy, Zechmeister, & Zechmeister, 1999). A typical introductory statis¬ 
tics text covers such topics as central tendencies, variability, z scores, probability, sam¬ 
pling distributions and hypothesis testing, and various parametric and nonparametric sta¬ 
tistics (Gravetter & Wallnau, 2004). Introductory statistics texts seldom show how a sta¬ 
tistic is prescribed by the methods chosen, and introductory methods books rarely show 
how methods limit the statistics used. Although combined research methods and statis¬ 
tics books do some integration of methods and statistics topics, such books are typically 
divided into large methods and statistics sections (e.g., Heiman, 2001). 

The compartmentalized approach is maintained as teachers’ curricula often mirror that of 
the textbooks they use. Indeed, one could argue that the novelty and the difficulty of 
methods and statistics concepts require addressing the topics one by one. Add students’ 
anxiety about statistics and methods to the newness of the topics (Onwuegbuzie, 2004), 
and this segmented approach seems logical, if not necessary. 
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The downside of this approach is that students often fail to see the big picture. Although 
they may understand the different aspects of the research process, they do not always un¬ 
derstand how these seemingly disparate pieces fit together. Thus, one of the many diffi¬ 
culties in teaching statistics and research methods is helping students appreciate how sta¬ 
tistics and research methods connect within the research process as a whole. 

Another difficulty is actively engaging students in statistics and methods. We know that 
students benefit from active learning (Chickering & Gamson, 1987) and that they prefer it 
(Bonwell & Eison, 1991). Furthennore, teachers acknowledge that actually conducting 
research is essential to students’ understanding of statistics and research methods (Smith, 
1998; Snee, 1993; Yoder, 1979). The exercise I present actively engages students in re¬ 
search by having them conduct it. Through the exercise, students experience the research 
process from start to finish in a single class period, and thereby experience the big pic¬ 
ture. 

In addition to providing a global view of the research process, the exercise meets two 
other objectives. One is the opportunity to experience and solve the problems that a re¬ 
searcher encounters while engaged in research (Smith, 1998). Introductory textbook de¬ 
scriptions of research and teachers’ examples do not always convey the mundane difficul¬ 
ties one experiences in devising and conducting research. However, experienced re¬ 
searchers know that problems such as sampling errors or missing data are part of the re¬ 
search enterprise and that mistakes such as poorly operationalizing variables or collecting 
unusable data are common for beginning researchers. In their first efforts, students inevi¬ 
tably make mistakes that they must try to rectify (Felder & Brent, 2003). By encounter¬ 
ing and overcoming obstacles in the research process, students actively leam lessons, and 
lessons actively learned are lessons remembered (Chickering & Gamson, 1987; Keeler & 
Steinhorst, 1995). 

The other objective of the exercise is to develop the seeds of mastery. Because students 
have learned about research without doing it, they have no reason to feel efficacious — to 
believe that they, too, can be researchers. Many approach their statistics and methods 
courses with trepidation that lingers (Onwuegbuzie, 2000, 2004). Even those who be¬ 
come confident in their understanding of the individual concepts may think themselves 
incapable of successfully engaging in the entire enterprise. This exercise provides an op¬ 
portunity for success. Students and their classmates experience all the basic steps of a 
research project, and in so doing, discover that they all can be researchers. 

Method 

I taught a research methods course that was the second quarter of a two-quarter statistics 
and methods sequence. Although students had exposure to statistics and pieces of the 
research process, they had not yet put the pieces together. To address this shortcoming, I 
developed an exercise that gave students the opportunity to experience the entire research 
process in a single period. I conducted the exercise on the first day of the research meth¬ 
ods course. 


The Journal of Effective Teaching, Vol. 8, No. 1, 2008,13-20 

: 2008 All rights reserved 



Experiencing the Research Process 


15 


Procedure 

I randomly assigned students to work in groups of three. Randomly assigning students 
provided the opportunity to review random assignment (vis-a-vis random sampling), and 
prevented self-selection into groups of distinctly different ability levels (see Gumming, 
1983). Though students often have mixed feelings about group work, I emphasized that 
the research process is almost always collaborative. 

Each group developed three testable research questions, with the ultimate goal of choos¬ 
ing one as the question that they would answer. The criterion for the questions was sim¬ 
ple: Students were to ask questions for which they could collect empirical data on or 
around campus in the next hour. They were to state their chosen question clearly in a sen¬ 
tence. If the question implied a relation among variables, then they were to state the 
question in the If: Then: Than format of a hypothesis. One such question was, “If people 
enter the east door of the student union, then they will be more likely to buy a coffee than 
people who enter through the west door.” Some other questions were as follows: “If 
people grew up in a metro area, then their political views will be more liberal than those 
of people who grew up in rural areas.” “Most people know someone who has been diag¬ 
nosed with depression.” Students in my course had all had introductory psychology 
where they learned the scientific method. Most also had had one or more content courses 
(e.g., abnormal, developmental, social) and knowing the content helped them come up 
with psychological, interesting questions. However, the hypotheses did not necessarily 
have to relate to psychology per se. Another example question was, “How many build¬ 
ings are there on campus?” At first blush, a question about buildings seems absurdly sim¬ 
ple, but many methodological issues arose when answering it (e.g., operational defini¬ 
tions of “building” and “on campus”). 

Students had approximately 8 min to develop their potential questions (Step 1). The time 
allocations are suggestions, and are laid out in Table 1. With the exceptions of bringing 
students together for step 2 and the final discussion, students typically worked through 
the steps at their own pace, spending more time than suggested on some steps and less on 
others. 

After time had elapsed, students met as a class, and each group proposed one of their 
three hypotheses (Step 2, 15 min). The entire class critiqued the hypotheses and made 
suggestions for improvement. Often these criticisms and suggestions concerned practi¬ 
calities: How was the group defining the variables? How were they going to gather and 
track their information? When their favored question provided too many stumbling 
blocks, groups turned to one of their other questions. I facilitated the discussion but pro¬ 
vided little direction, letting students supply the feedback, and letting each group settle on 
the single question they wished to answer. Although potential problems were evident 
(e.g., inclusion of a hard-to-define variable), I chose to let students proceed and discover 
their problems and solutions through the process. In classes where students are not yet 
familiar with research ethics, the teacher may have to guide question selection more. 
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Table 1. Steps in the Research Exercise 


Step 

Activity 

Time Allowed 

Step 1 

Question development 

8 min 

Step 2 

Hypothesis selection 

15 min 

Step 3 

Operationalize variables 

5 min 

Step 4 

Method development 

5 min 

Step 5 

Data collection 

20 min 

Step 6 

Data entry and analysis 

10 min 

Step 7 

Results interpretation 

10 min 

Step 8 

Preparation of brief report 

10 min 

Step 9 

Sharing with the class 

20 min 


Note. The activities and times indicated are suggestions. The 


instructor may modify the steps and time allotments as they choose. 


After selecting a question, groups solidified how they were going to operationalize the 
variables of interest (Step 3, 5 min). As they thought about how to measure the variables, 
students thought about how they would track and analyze the data they intended to col¬ 
lect. During this and subsequent steps I was available for guidance, but encouraged stu¬ 
dents to work with each other. 

Students then devised their plan for gathering their infonnation (Step 4, 5 min). They 
pilot tested or tried out their methods on other groups and made adjustments as time al¬ 
lowed. 

In the fifth step (20 min), students gathered the information using the methods they had 
decided upon in the third and fourth steps. Some groups were able to collect their data 
from people in class and the surrounding area*. Other groups needed to go to specific lo¬ 
cations on campus. 

Two members of the group were to serve as the information gatherers, and the third was 
to “observe the observers.” Although everyone was to conscientiously record their ob¬ 
servations, the observer of the observers acted as the recorder of the process and docu¬ 
mented issues or problems that arose. He or she was also assigned to critique the infor¬ 
mation gatherers and note any concerns that emerged. I specified the time that students 
were to return to the classroom. 

After data collection, students brought their data to the computer laboratory for analysis 
(Step 6, 10 min). Most students used the SPSS program available on the computers in 


‘ I consulted my university’s Internal Review Board before conducting my research exercise. The IRB 
determined that further IRB approval was not necessary because the data were for in-class use only. If 
there are IRB concerns at other universities, students in fields that work with human subjects can collect 
their data by surveying or observing classmates. 
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our laboratory. For classrooms without student computers, one or two laptops with ac¬ 
cess to statistical software or a computer spreadsheet program could be provided. Alter¬ 
natively, the teacher could limit the studies to designs where complex statistical analysis 
is not necessary. 

In Step 7 (10 min), students interpreted their results, answering their questions. They 
were encouraged to present a table or graph if they thought that it would help the reader 
understand their findings. 

Students then incorporated their results into a short research report (Step 8, 10 min). 
Each report included the question the students chose and why, their clearly stated hy¬ 
pothesis, how they gathered and tracked their information, how they analyzed their data, 
the results, and their conclusion (i.e., the answer to their question or hypothesis). Of key 
importance was the final paragraph in which students described the issues that arose dur¬ 
ing their study: What were the problems that that they encountered and their solutions? 
What did the "observer of the observers" notice? How might they do things differently if 
they were to repeat the study? 

After students submitted their reports, the class reconvened to talk about their research 
(Step 9, 20 min). This shared reflection turned out to be a very important part of the 
process, as students talked about their problems and solutions and learned how their 
classmates overcame theirs. My role was to lead the discussion, making connections be¬ 
tween groups’ experiences, reminding them of definitions, and suggesting alternative so¬ 
lutions. 


Results 

One week later, students (N = 15) completed a short questionnaire. Students responded 
to the 9 items on Likert scales, typically ranging from 1 (not at all) to 7 (very). Means 
and standard deviations for all survey questions appear in Table 2. 

A dependent samples t test comparing their self-reported understanding before the exer¬ 
cise to their understanding after the exercise was highly significant, t( 14) = -5.292, p < 
.001, Msefore = 4.53 (579Before = 1-19), M A ft er = 5.87 (SD At \ a - = .74). Students found the ex¬ 
ercise to be both interesting and useful. Furthermore, students recommended that the ex¬ 
ercise be used in this course in the future and in courses at other universities. 

Students’ comments support the impact and value of the exercise. “I found the project to 
be challenging and also fun and interesting.” “Broke down the steps/processes well.” 
“Exposed me to the research process, very hands on and applicable knowledge.” “Did a 
great job applying all the concepts we learned last quarter.” “I feel like I have started a 
good background in research.” 
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Table 2. The Means and Standard Deviations of Participants’ Responses to Survey 
Questions 


Survey Questions _ 

How interesting did you find the exercise? 

How useful was the exercise in helping you understand the research process? 

How would you rate your understanding of the research process before the exercise? 
How would you rate your understanding of the research process after the exercise? 
How helpful was it to have class help you evaluate your potential questions? 

How useful was it to have the "observer of the observer?" 

How useful was the practice of operationalizing the variables? 

Would you recommend that this exercise be included in this course in the future? 
Would you recommend this exercise to professors at other universities? _ 


M D 


5.93 

1.10 

5.47 

0.92 

4.53 

1.19 

5.87 

0.74 

5.73 

1.28 

4.93 

1.59 

6.00 

0.76 

6.40 

1.06 

6.27 

1.10 


Note. Students responded on 7-point Likert scales from 1 {not at all ) to 7 {very). Stu¬ 
dents’ understanding of the research process after the exercise significantly differed 
from their understanding before the exercise,/; < .001. 


Discussion 


Taken together, the results suggest that the students found the exercise to be valuable, for 
themselves and potentially for other students and professors. Not all students cared for 
the “observer of the observers” (see Table 2). It is one thing to provide feedback along 
with others in class and another to be the one criticizing one’s group members. In future 
iterations of the exercise, the observers recorded everything they saw, but did not record 
their evaluations of their observations. In discussion, reporting what they saw revealed 
the problems, while taking away the sting of criticism. 

The class that participated in this initial exercise was small and the class period was long 
(2 h). I have since successfully replicated the exercise with larger classes and with 
shorter class periods. With larger classes, four or five students can be placed in small 
groups, or several groups can be randomly selected to present their potential research 
questions or final results to the class. 

With shorter classes, I placed limits on the steps or omitted steps altogether. For exam¬ 
ple, I limited students to producing and sharing one research question, limited research 
questions to those that can be answered in the classroom by surveying or observing 
classmates, or limited the time allotted for the steps. In other short classes, students 
orally presented their results, bypassing the creation of written reports. For classes short 
on time, statistical knowledge, or computer access, students can provide only descriptive 
results such as frequencies or averages from their studies. I have found that having a 
somewhat limited amount of time is actually beneficial in that it induces students to work 
quickly and efficiently. 

In longer classes, more time can be allotted to the steps and professional reporting stan¬ 
dards (e.g., American Psychological Association style) can be expected in the report. 
The exercise can also be stretched across two class periods, with question development 
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through data collection on one day (steps 1 through 5), and data analysis, interpretation 
and presentation on the next (steps 6 through 9). 

Although the class that gave feedback on this exercise already had a large knowledge 
base (over a quarter of statistics), I have used this exercise on the very first day of class of 
a two-quarter statistics and research methods sequence and a colleague used it success¬ 
fully at the beginning of a statistics course (L.S. Wagner, personal communication, No¬ 
vember 7, 2006). Students (alone or in pairs) follow the steps, skipping steps 2 (initial 
class feedback) and 6 (SPSS). They gather data from their classmates and in the end re¬ 
port what they have found. Students get an overview of the research process, make mis¬ 
takes that they are unlikely to make again, and begin to feel like the researchers they can 
be. 

The exercise is not meant to be students’ only experience with the research process. Ide¬ 
ally, students would experience the research process often, and occasionally on a larger 
scale, where they investigate questions in the extant literature, derive appropriate meth¬ 
odology for their hypotheses, seek Internal Review Board approval, select appropriate 
samples, and so on until completing professionally formatted manuscripts. Although 
some undergraduate programs provide research experiences for all of their students, 
many do not. In addition, the majority of undergraduate programs do not have integrated 
research methods and statistics courses (Messer, Griggs, & Jackson, 1999). For those 
programs that do not offer full research experiences and integrated methods and statistics 
courses, an exercise like the one I offer allows students to experience the whole research 
process. Professors are encouraged to adapt the exercise to their class topic, size, and 
time, and to repeat the exercise as their schedule allows. As students’ understanding be¬ 
comes more sophisticated across the quarter, so do their questions and statistical and 
methodological approaches. 

Although this exercise was designed and tested in psychology statistics and research 
methods courses, the outline of the process is applicable to statistics and methods courses 
in any subject area. Indeed, questions need not be domain-specific for the process to be 
meaningful. In analogous areas (i.e., the social sciences), students working in groups can 
create testable questions, or the seeds of questions that the class can help them develop, 
from the content learned in introductory courses. In the “hard” sciences (e.g., physics), 
teachers may want to provide an array of basic equipment around which questions should 
be devised. In any discipline, a teacher may want to circumscribe the questions or the 
data sets that students can use. Providing a list of possible topics to pursue or data sets 
around which to develop questions affords some limits while allowing students choice. 

However other teachers choose to modify the exercise, I encourage them to maintain the 
suggested beginning and end - allowing students to choose their questions and having 
students share what they leam. I believe that encouraging students to pursue their own 
ideas leads to greater investment in the exercise. In my statistics and methods courses I 
use a variety of class experiments and activities that, while successful, never engender the 
excitement that this approach to the research process does. I believe that the other key 
piece to the success of this exercise is discussing what is learned, both from the studies 
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conducted and from the exercise. The process of sharing provides a chance for students 
to realize what they have learned and to learn from the mistakes and successes of others. 

Incorporating exercises that actively engage students in the entire research process com¬ 
bats students’ compartmentalized conceptions of research and shows them the big pic¬ 
ture. In addition such exercises afford opportunities for students to experience and solve 
the problems that researchers encounter, and in the end, to discover that they all can be 
successful researchers. 
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